Skip to content

Fix MIL syntax + M1/M2 support - #6

Open
imperatormk wants to merge 4 commits into
maderix:mainfrom
imperatormk:fix/mil-scalar-syntax
Open

Fix MIL syntax + M1/M2 support#6
imperatormk wants to merge 4 commits into
maderix:mainfrom
imperatormk:fix/mil-scalar-syntax

Conversation

@imperatormk

Copy link
Copy Markdown

MIL scalar types used shorthand syntax that only compiles on M4. Changed to the canonical verbose format that CoreML's compiler emits (tensor<string, []>("x") instead of string("x")), targets program(1.0)/<ios16>, and simplifies buildInfo.

For conv kernels, adds runtime fp16 I/O fallback — M1/M2 ANE can't do the cast op, so on first compile failure it retries with fp16 inputs/outputs and converts on the CPU side.

Tested on M1 Pro, macOS 26.3.

The MIL scalar types used shorthand syntax (string("x"), int32(1)) that
only works on M4. Changed to the canonical verbose format that CoreML's
own compiler emits (tensor<string, []>("x"), tensor<int32, []>(1)).

Also targets program(1.0) with <ios16> instead of program(1.3)/<ios18>,
and simplifies buildInfo to just coremlc-version.

For conv-based kernels, adds runtime fp16 I/O fallback — M1/M2 ANE
doesn't support the cast op (fp32<->fp16), so on first compile failure
it retries with native fp16 inputs/outputs and does the conversion on
the CPU side. The fallback is persisted across exec() restarts.

Note: matmul and scaled_dot_product_attention ops still fail on M1/M2 —
these are M4+ ANE ops. The attention tests (test_ane_causal_attn,
test_ane_sdpa5, test_full_fused attention part) require M4 hardware.
Conv-based kernels (training, QKV projections, FFN) work on all generations.

Tested on M1 Pro, macOS 26.3 (Tahoe).
@abhishekgahlot2

abhishekgahlot2 commented Mar 2, 2026

Copy link
Copy Markdown

you need to fix the other files too not just the training dir files to make it work on m1/m2.

@imperatormk

Copy link
Copy Markdown
Author

From what I see should be complete now although I'd like to fix attn too, would be keen to train a nice UNet like this

codegen-sh Bot referenced this pull request in dermitchell1993/ANE Mar 2, 2026
Port upstream PR #6 (imperatormk) - fixes MIL scalar type syntax
from M4-only shorthand to canonical verbose format that compiles
on all Apple Silicon (M1/M2/M3/M4).

Changes:
- program(1.3) to program(1.0), ios18 to ios16 target
- Scalar type shorthand to canonical verbose format
- Simplified buildInfo dict (no M4-specific version strings)
- fp16 I/O fallback: g_fp16_io flag with auto-retry on compile
  failure for M1/M2 where cast op is unsupported
- Dynamic IOSurface byte calculation (bpe: 2 for fp16, 4 for fp32)

Tested on M1 Pro, macOS 26.3 (per upstream PR author).
dev-erik added a commit to dev-erik/ANE that referenced this pull request Mar 3, 2026
…program(1.0), ios16 target, tensor types across 18 files
ebowwa pushed a commit to ebowwa/ANE that referenced this pull request Aug 4, 2026
… hardening

#1 /models bypassed safety: now uses services.safe_compile (lint → validate → passes → compile)
instead of calling bridge.compile directly. The shared compile service enforces all /raw/kernels
safety rules (MIL lint, positive tensor-size validation, tensor-count limits, compiler passes,
weight-name validation, bridge-error mapping).

#2 Model leases leaked: register_model now does atomic replace (free old lease before installing
new). unregister_model calls bridge.free(kernel_id) before removing the dictionary entry.

maderix#3 Async eval bypassed admission: now calls admit_eval BEFORE dispatching the background task,
and finish_eval in _bg's finally block. max_pending backpressure applies to async work.

maderix#4 Cancellation resurrection: _bg checks sub.state == CANCELLED before transitioning to EXECUTING.
A cancelled submission is not resurrected.

maderix#5 Submission memory leak: SubmissionRegistry now has max_retained=100 + ttl_s=300. _prune()
evicts terminal submissions by TTL and count on every create().

maderix#6 Batch-eval bypassed admission: each item now goes through admit_eval/finish_eval. Endpoint
honestly named "serial batch" (NOT parallel without kernel replication).

138 tests.

Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa pushed a commit to ebowwa/ANE that referenced this pull request Aug 4, 2026
…models admission

#2 W&B: _process now handles event/run_finish (previously discarded). Fallback delegates
correctly when wandb absent (metrics + traces + events forwarded). finish_run finalizes in
queue order (no race). flush() drains with 5s timeout.
maderix#3 LocalJSON: writer calls task_done() (flush no longer deadlocks).
maderix#4 Backend registry: constructed in lifespan (ANEBackend + MLXBackend), stored on
app.state.backends.
maderix#6 /models: run_model now goes through admit_eval/finish_eval (previously bypassed
scheduler). register_model uses safe int parsing (try/except → 400, not 500).
maderix#8 /v1/resources: uses app.state.settings (was always false), ANE_PUBLIC_ENDPOINT env
(was guessing hostname), correct tailscale_serve detection. Lifespan sets
app.state.settings.
Lifespan flushes all observability backends on shutdown.
mlx has platform marker in requirements.txt (Darwin arm64 only).

149 tests.

Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa pushed a commit to ebowwa/ANE that referenced this pull request Aug 4, 2026
…, W&B fallback, deadline parsing

#1 All integration routes (/mlx/compare, /mlx/diff, /pipeline) now go through admit_eval /
finish_eval — no route bypasses the scheduler. Pipeline ANE steps also admit. The /mlx/compare
demo uses try/finally to guarantee kernel lease release.

#2 Batch-eval validates inputs_per_item >= 1 and rejects input counts not exactly divisible
(no silent tensor discard). Both deadline_ms parses (eval + batch) now use try/except → 400
for non-numeric X-Deadline-Ms (was 500).

maderix#3 W&B fallback: when wandb is not installed, log_metrics / log_event / log_trace delegate
directly to the fallback instead of queueing into an unconsumed queue (previous code started
no uploader thread without wandb → events stuck forever).

maderix#4 SubmissionRegistry.mark_terminal() calls _prune() immediately when a submission completes
(no waiting for the next create()).

maderix#6 test_models_register_rejected_by_lint: skips when ANE bridge unavailable (was asserting
422 on non-ANE hosts where /models returns 503 before reaching lint).

Pipeline spec JSON errors now return 400 (was 500). 149 tests.

Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa pushed a commit to ebowwa/ANE that referenced this pull request Aug 4, 2026
… mx.distributed, full output descriptors

maderix#6 Remote store qualified references: put/get/list/delete now use tier-prefixed IDs
(remote:<host>:<tid> or local:<tid>). A failed remote put that falls back to local returns
a local: ID; get/list/delete route to the correct tier based on the prefix. list() merges
both tiers. The namespace is coherent — no silent namespace collision between machines.

maderix#7 Deleted stale mx.distributed path: removed ane_distributed.py, mlx_worker.py, and
test_distributed.py. mx.distributed only supports single-machine multi-core; the TCP worker
(mlx_tcp_worker.py + ane_tcp_transport.py) is the real cross-machine path. The distributed
endpoints were already removed in the prior commit; now the dead code is gone.

maderix#8 ANE backend output descriptors: load() now stores full output_descriptors from the
ArtifactDescriptor spec (dtype + shape + byte_length) per executable_id. execute() constructs
output TensorDescriptors from the stored specs, not invented [1,N,1,1] shapes. unload() clears
the specs. Callers who provide output_descriptors in the artifact spec get exact output
metadata; callers who don't get byte_length-correct descriptors with empty shape.

7 tests (updated for qualified refs + merged list + fallback). 156 total.

Co-Authored-By: Claude <noreply@anthropic.com>"
git push -q origin ane-compute-api && echo "pushed"; git rev-list --count origin/main..HEAD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants